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ABSTRACT 

In higher education settings more and more learning is facilitated through online learning environments. To support and 
understand students’ learning processes better, learning analytics offers a promising approach. The purpose of this study 
was to investigate students’ expectations toward features of learning analytics systems. In a first qualitative exploratory 
study a total of 20 university students participated. They were interviewed about their expectations of learning analytics 
features. The findings of the qualitative study were validated in a second quantitative study in which 216 students took 
part. Findings show that students expect learning analytics features to support their planning and organization of learning 
processes, provide self-assessments, deliver adaptive recommendations, and produce personalized analyses of their 
learning activities. 


KEYWORDS 

Learning analytics; higher education; students’ expectations; self-regulated learning 


1. INTRODUCTION 

Higher education institutions develop and implement learning analytics systems to support student learning. 
Therefore, it is relevant to consider students’ expectations of such systems in terms of learning. Learners 
directly interact with the user interface of the learning analytics system, which offers different features such 
as visualizations, learning recommendations, prompts, rating possibilities, and self-assessments. Further, 
learning analytics systems aim to offer highly adaptable and personalized learning environments (Ifenthaler 
& Widanapathirana, 2014). Personalized learning environments can help to foster students’ skills to manage, 
monitor, and reflect their own learning (McLoughlin & Lee, 2010). To be able to design, develop, and 
implement personalized learning analytics systems, it is necessary to investigate what learners expect from 
these systems. Otherwise the implementation of learning analytics systems as a means to support learning 
could fail as it might even hinder self-regulated learning for example if students feel demotivated because of 
their performance in comparison to their peers. Conole, Creanor, Irving, and Paluch (2007) showed in their 
study on e-learning that it is necessary to recognize a hill range of students’ perceptions as otherwise 
institutions might fail to meet learners’ needs. 

Still, empirical research regarding students’ expectations on learning analytics to facilitate learning is 
scarce. Therefore, the purpose of this mixed-methods study was to investigate which features of learning 
analytics systems students expect. To validate the findings of a first exploratory qualitative study, a follow-up 
quantitative study was conducted to investigate how students rate learning analytics features in terms of 
learning and their potential implementation. 
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2. LEARNING ANALYTICS FEATURES 

2.1 Learning Analytics 

Learning analytics use static and dynamic information about learners and learning environments, assessing, 
eliciting and analyzing them, for real-time modeling, prediction, and optimization of learning processes, 
learning environments, and educational decision-making (Ifenthaler, 2015). 

Learning analytics provide benefits for all levels of higher education stakeholders: mega-level 
(governance), macro-level (institution), meso-level (curriculum, teacher/tutor) and micro-level (learner) 
(Ifenthaler & Widanapathirana, 2014). The micro-level of learning analytics focusses on supporting 
individual and collaborative learning activities. Benefits can be divided in three perspectives and for the 
micro-level as follows (Ifenthaler & Widanapathirana, 2014): (a) summative: understand learning habits, 
compare learning paths, analyze learning outcomes, track progress towards goals; (b) real-time: receive 
automated interventions and scaffolds, take assessments including just-in-time feedback, support 
collaboration; (c) predictive: optimize learning paths, adapt to recommendations, increase engagement, 
increase success rates. 

2.2 Learning Analytics Features 

Learning analytics features include functions a learning analytics system could provide to the user (e.g., 
learners, tutors, administrators, etc.). Thus learning analytics features include dashboard elements, for 
example visualizations of activity analyses in the learning management system. Further, features include 
recommendations about further readings, self-assessment-questionnaires, or additional links to related video 
tutorials. Features focusing on learners’ behavior include time spent online, analyses and forecasts of 
academic performance, adaptive learning recommendations, and personalized prompts with questions about 
the learners’ dispositions. 

Learning analytics features rely on analyses of various data (Ifenthaler & Widanapathirana, 2014): (a) 
Learner characteristics including prior knowledge, psychometric tests about learning strategies and 
competencies, socio-demographic data, or prior academic performance, (b) External data such as searches in 
the library catalogue, geo-data or information from social media, (c) Traces generated by using the online 
learning environment, for example online-frequency and -time, activities in discussions and other online 
interaction, results of self-assessment-questionnaires, up- and download of resources, as well as ratings of 
content. Furthermore, (d) curricular information are integrated into the analyses, for instance exemplar study 
paths and expected learning outcomes. 

Many dashboard applications in learning analytics systems focus on visualizations of descriptive data, 
such as time spent online, the progress towards the completion of a course, or comparisons with other 
students’ performance. More elaborated systems include results of self-assessments (Verbert et al., 2014). 
Findings of a comparative study of three learning analytics systems have shown that students prefer more 
detailed learning analytics systems with elaborated analyses and personalized recommendations for their 
learning (Ifenthaler & Schumacher, 2015). 

Learning analytics systems showing descriptive summative information about past learning activities such 
as time spent online, login frequency, and performance results already help students to monitor their current 
state and increase student success rates.. However, to plan upcoming learning activities or to adapt current 
learning strategies, further personalized and adaptive features of the learning analytics system are needed. 

2.3 Purpose of the Studies 

Currently, learning analytics features are investigated in terms of visualizations and dashboard elements or 
their technical possibilities (Verbert, Duval, Klerkx, Govaerts, & Santos, 2013; Verbert et al., 2014). Most 
studies about learning analytics dashboards were conducted in controlled settings. 

To better understand students’ needs, this study used a mixed-methods approach combining an 
exploratory qualitative interview study followed by a quantitative survey study. The two studies are reported 
in the following sections. 
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3. STUDY 1 

The purposes of the qualitative exploratory study were ( 1 ) to investigate which features of learning analytics 
students expect and (2) to deduce further research needs. 

3.1 Method 

3.1.1 Participants and Design 

The study was designed as a qualitative exploratory study with oral interviews. Interviews were conducted in 
May 2016. After removing one incomplete response, the responses of 20 graduate students (14 female, 6 
male) have been considered for further analyses. The average age of the participants was 24.55 years ( SD = 
2.21 ). Participants received one credit hour for participating in the study. 

3.1.2 Materials 

Introduction to Learning Analytics 

A short lecture (approximately 5 minutes) including presentation slides introduced the basic concepts of 
learning analytics and provided an overview about various types of data used for learning analytics. The 
session concluded with a possibility to clarify comprehension questions. 

Learning Analytics Features 

Students were confronted with three guiding questions regarding learning analytics features, which they were 
asked to answer in oral form or by using a whiteboard or paper to illustrate. (1) Please reflect about possible 
features or dashboard elements, which you would like to have as an application in a learning analytics 
system. (2) Please explain which function these elements have. (3) Please indicate how you think these 
features or dashboard elements can support learning. 

Technology Usage for Learning 

The Technology Usage for Learning (TUL; 10 items) inventory includes items investigating students’ usage 
and attitude towards technology for learning purposes and the potential use of learning analytics systems. 

Demographic information 

Demographic information included age, gender, Internet usage for learning and social media, years of study, 
study major, and course load. 

3.1.3 Procedure 

Over a period of two weeks in May 2016, students were invited to participate in the qualitative interview 
study, which included three parts. In the first part, participants received a general introduction into learning 
analytics (approx. 5 minutes). Secondly, they reported about learning analytics features, they would expect 
from learning analytics systems and how these features could support learning (8 up to 80 minutes). In the 
third part, participants completed the technology usage for learning inventory and reported their demographic 
information ( 10 minutes). 

3.1.4 Data Analysis 

The audio recordings of all interviews were transcribed in single text documents. Using f4anaylsis 
(www.audiotranskription.de), a software for qualitative data analysis, the transcribed interviews were 
analyzed in terms of learning analytics features and critical statements the students mentioned. Afterwards 
the collected features were parsed again to find out which features tended to be more relevant for the 
respondents but following the qualitative research approach also single statements were considered. 
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3.2 Results 

For the majority of the students a learning analytics system would help in terms of planning their learning 
activities and higher education studies. The students’ demands ranged from basic reminder timet ions for 
deadlines, for example to submit assignments towards automated to-do lists and agendas. Two interviewees 
(4, 21) would also allow the system to have access to their own calendar and recommend appropriate tasks 
and plans matching their personal schedule. 

The majority of the participants mentioned that the system should offer self-assessments corresponding to 
their learning fields. The assessment conditions should be the same as during exams regarding working time 
and task difficulty. Ideally after completion they want to receive direct and valid feedback. The feedback 
should be divided into subject areas enabling students to assess their need for improvement. To initiate 
further learning activities, the system should provide corresponding learning recommendations such as 
further material. In general, they expect further learning material presenting the same content with different 
media such as videos, lecture recordings and short summaries or beyond that on-topic links to current news, 
practical examples, further literature or learning materials of previous courses to recapitulate. 

An overview about their current state of knowledge, their activities in the system as well as their progress 
towards own or set learning objectives seems to be relevant to the students. They asked for analyses of their 
working progress towards learning objectives, time spent for learning, preferred daytime for learning, 
performance and progress over different periods as well as performance evaluations of former and current 
grades and also forecasts. 

Students disagreed in terms of receiving analyses comparing their own performance or learning activities 
with those of their peers. Some perceive such comparisons as motivating others would not. To avoid 
demotivation and to meet all learners’ needs it was suggested to have a high degree of personalization 
regarding the system’s visualizations or that they should only be shown on request. 

Personalization options were also requested regarding the layout of the learning environment in terms of 
colors and disposal of elements. 

Interaction with fellow students but also direct contact to the lecturer via the system was essential for the 
students. Discussion forums and chats for communication as well as videoconferencing and online -teamwork 
function with the possibility to share documents were mentioned features. Additionally, two interviewees (1, 
11) wanted the system to suggest learning partners, which are either close by, are dealing with the same 
learning subject or have complementary knowledge to create synergies. 

Statements from only few or single students uttered the possibility to click on keywords in all provided 
materials, which would lead to definitions or further learning material concerning these keywords. Further it 
was argued that the system needs a feature to enter learning activities occurring offline. This aspect becomes 
even more relevant as the additional questionnaire revealed that all interviewed students prefer to read 
printed texts for their studies, only one student additionally reads texts on a screen. 

Students’ statements also indicated that they expect a highly evolved system, containing several 
programs, such as text processing, a literature management program being fed by the library, or a PDF 
annotation program. The students stated that they do not want to switch between programs but would prefer a 
holistic solution. 

Two students (4, 20) would like the system to guide their breaks by analyzing their productivity and 
suggesting when it is time for a break. Further the system might give advices which kind of break would be 
reasonable, for example eating, drinking, or doing distracting activities (watching a video or doing a 
workout). 

Most interviewees had a positive attitude regarding the application of learning analytics systems and 
would like to use such a system. Only two respondents indicated in the questionnaire that they do not want to 
use learning analytics, due to privacy concerns, the risk of too much surveillance of learning activities and a 
reduction of autonomous learning (9). Flowever, also the students who agreed to use learning analytics had 
critical thoughts as for instance demotivating consequences due to visualization of poor performances or 
comparisons with fellow students or the distractive character of using media and technology for learning. 
One interviewee feared that even stronger as yet not acquiring knowledge but figures such as good grades are 
the only purpose of learning. 
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4. STUDY 2 

Based on the findings of the qualitative study, a follow-up quantitative study was conducted. The first 
assumption was that the learning analytics features presented to the students were rated differently in terms of 
students’ willingness to use the feature for their learning (Hypothesis 1). Second, it was assumed that 
students’ evaluation of the presented features in terms of learning differed significantly (Hypothesis 2). 
Finally, it was assumed that students are more willing to use a certain learning analytics feature for their 
studies when rating the feature high in terms of learning (Hypothesis 3a). do not perceive that the feature is 
invasive (Hypothesis 3b), and do not think that the feature is complicated to use (Hypothesis 3c) or not useful 
for them (Hypothesis 3d). 

4.1 Method 

4.1.1 Participants and Design 

The second study was designed as a quantitative online study conducted in May and June 2016. The average 
age of the participants was 23.83 years (SD = 2.99). The dataset included N = 216 responses (142 female 
(66% valid) and 73 males (34% valid) [1 missing]). More than half of the participants studied in the 
Bachelors program (54,6%) and 45,4% students studied in the Masters program. The average course load in 
the current semester was 5.42 courses (SD = 1.91). Almost half of the students (46%) indicated that they 
prefer reading texts for university on a display whereas 54% preferred reading printed texts. 88% of the 
interviewed students want to use learning analytics for their studies, whereas 12% did not want to use 
learning analytics. Participants received one credit hour for participating in the study. 

4.1.2 Instruments 

Learning Analytics Features (LAF) 

The participants were confronted with 15 different learning analytics features, some of them deduced from 
the qualitative exploratory study: (1) time spent online; (2) suggestion of learning partners; (3) learning 
recommendations for successful course completion; (4) rating scales for provided learning material; (5) 
timeline showing current status and goal; (6) time needed to complete a task or read a text; (7) prompts for 
self-assessments; (8) further learning recommendations; (9) comparison with fellow students; (10) 
considering the students personal calendar for appropriate learning recommendations; (11) newsfeed with 
relevant news matching the learning content; (12) revision of former learning content; (13) feedback for 
assignments; (14) reminder for deadlines; ( 15) term scheduler, recommending relevant courses. 

The students were asked to rate these 15 features in terms of learning, acceptance, and privacy aspects. 
All items were answered on a 5 -point Likert scale (1 = strongly disagree; 2 = disagree; 3 = neither agree or 
disagree; 4 = agree; 5 = strongly agree; LAF; 20 items; Cronbach’s a = .93). 

Learning Analytics Benefits (LAB) 

The learning analytics benefits scale (LAB) focuses on benefits, learning analytics could offer (Ifen thaler & 
Widanapathirana, 2014). The students were asked to rate the 36 items on a 5-point Likert scale (1 = strongly 
disagree; 2 = disagree; 3 = neither agree or disagree; 4 = agree; 5 = strongly agree) (LAB; 36 items; 
Cronbach’s a = .94). 

Privacy for Learning Analytics (LAP) 

In the privacy for learning analytics questionnaire (LAP) the students were asked to state their willingness to 
share personal data for learning analytics systems, for example tracking of their online paths, educational 
history, course of studies etc. All items were answered on a Thurstone scale ( 1 = Agree; 2 = Do not agree; 
LAP; 23 items, Cronbach’s a = .84). 

Self-Regulated Learning Scale (SRLS) 

To assess students’ capability to self-regulate their learning the adapted version of the “Inventory of Learning 
strategies” (Boerner, Seeber, Keller, & Beinborn, 2005; Wild & Schiefele, 1994) was used and adjusted. The 
final scale (SRLS) included 14 subscales: effort (8 items, Cronbach’s a = .76); concentration (6 items. 
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Cronbach’s a = .91); critical thinking (8 items, Cronbach’s a = .82); learning environment (6 items, 
Cronbach’s a = .73); metacognitive awareness (7 items, Cronbach’s a = .65); organization (9 items, 
Cronbach’s a = .81); regulation (8 items, Cronbach’s a = .72), help & resources (10 items, Cronbach’s 
a = .7); self-efficacy (4 items, Cronbach’s a = .64); self-assessment (7 items, Cronbach’s a = .67); revision (7 
items, Cronbach’s a = .72); time management (4 items, Cronbach’s a = .82); goal setting (7 items, 
Cronbach’s a = .71); coherence (8 items, Cronbach’s a = .69). All items were answered on a 5-point Likert 
scale (1 = strongly disagree; 2 = disagree; 3 = neither agree or disagree; 4 = agree; 5 = strongly agree; SRLS; 
99 items, Cronbach’s a = .93) 

Technology use for Learning Scale (TUL) 

The technology use for learning scale focuses on how often students use certain technologies and media (e.g., 
laptop, tablet, blogs, podcasts) for learning (TUL1; 11 items, Cronbach’s a = .65). And which technologies 
and media they would like to use more often for learning purposes (TUL2; 1 1 items, Cronbach’s a = .74). 

Demographic Information 

Finally, students stated demographic information such as age, gender, course load, Internet use, current 
academic performance (20 items). 

4.1.3 Procedure 

In May and June 2016 over a period of three weeks, students could participate in an online study, 
implemented on the university’s server and consisting of five parts. In the first part, students received a 
general introduction into learning analytics (approx. 5 minutes). The second part focused on learning 
analytics: The students rated the 15 learning analytics features, each by answering 20 items (LAF; 30 
minutes). Then, they completed the learning analytics benefits scale (LAB; 36 items, 15 minutes). And 
finally, they participated in the privacy for learning analytics questionnaire (LAP, 23 items, 10 minutes). In 
the third part, the students were confronted with the self-regulated learning scale (SRLS; 99 items, 25 
minutes). Afterwards, students reported which technologies they use and would like to use more for learning 
(TUL; 22 items; 10 minutes). Finally, participants reported their demographic information (20 items, 7 
minutes). 

4.2 Results 

4.2.1 Acceptance to use Learning Analytics Features 

Students’ rating, if they would like to use the presented learning analytics features for their studies differed 
significantly, /•’( 1 4,3225) = 48.069, p <.()() I , rj 2 = .173. Games Howell post-hoc comparisons between the 
different learning analytics features revealed that the reminder function (M = 4.2, 95% Cl [4.06, 4.34] was 
evaluated significantly higher than the newsfeed providing current learning content relevant news (M = 3.42, 
95% Cl [3.24, 3.61], p < .001 (see Table 1 ). 

The learning analytics function to repeat former learning content ( M = 4.12, 95% Cl [3.99, 4.25]) was 
rated significantly higher than the feature showing the time, which is necessary to complete a task or reading 
a text ( M = 2.32, 95% Cl [2.14, 2.51]), p < .001. Getting automated feedback for assignments ( M = 4.07, 
95% Cl [3.91, 4.22]) was rated significantly higher than the function that the system considers the learner’s 
personal schedule and gives matching learning recommendations ( M = 3.5, 95% Cl [3.33, 3.68]) p < .001. 
The rating of the features regarding the willingness to use a certain feature differed significantly, accordingly 
Hypothesis 1 is accepted. 
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Table 1. Post-hoc comparisons for item “would I like to use for my studies” by learning analytics features 

Feature Statistic 

1 Time online Mean 3.125 

Variance 1.422 

SD 1.1923 

2 Learning partner Mean 3.301 

Variance 1.272 

SD 1.1277 

3 Learning Mean 3.981 

recommendation for Variance 1.023 

course completion SD 1.0114 

4 Ratingscales for learning Mean 3.306 

material Variance 1.813 

SD 1.3465 

5 Timeline for Mean 3.778 

achievement of Variance 1.392 

objectives SD 1.1799 

6 Time for reading and Mean 2.324 

task completion Variance 1.848 

SD 1.3594 

7 Prompts for self- Mean 4.074 

assessment Variance .999 

SD .9996 

8 Further learning Mean 3.685 

recommendations Variance 1.352 

SD 1.1626 

9 Comparison with fellow Mean 2.491 

students Variance 1.739 

SD 1.3189 

10 Integration of personal Mean 3.5 

schedule Variance 1.693 

SD 1.3012 

11 Newsfeed Mean 3.421 

Variance 1.938 

SD 1.3921 

12 Repetition of learning Mean 4.12 

content Variance .878 

SD .9373 

13 Feedback for Mean 4.069 

assignments Variance 1.386 

SD 1.1772 

14 Reminder Mean 4.199 

Variance 1.137 

SD 1.0663 

15 Term scheduler Mean 3.667 

Variance 1.795 

SD 1.3399 

Note: *** pc.OOl; ** pc.Ol; * pc. 05 


4.2.2 Learning Analytics Features Rated for Learning 

After building a learning scale (14 items, Cronbach’s a = .94) and computing ANOVA it could be shown that 
the rating of the features in terms of learning differed significantly, /*’( 1 4,3225) = 56.49, p <.001, rf = .197. 
Games-Howell post-hoc comparisons (see Table 2) indicated that students evaluated prompts for 
self-assessments ( M = 3.73, 95% Cl [3.64, 3.82] significantly higher than feedback on assignments 
( M = 3.39, 95% Cl [3.27, 2.51]). Learning recommendations to complete the course ( M = 3.7, 95% Cl [3.61, 
3.77]) were rated significantly higher than suggested learning partners (M = 3.13, 95% Cl [3.03, 3.23]). 
Students evaluated a feature showing a timeline with their status quo towards their objectives (M = 3.63, 95% 
Cl [3.53, 3.73]) significantly higher than information about their time spent online (M = 2.99, 95% Cl [2.88, 
3.1]). Hence, students’ evaluation of the presented features concerning learning differed significantly, 
accordingly Hypothesis 2 is accepted. 


1 2 3 4 5 6 7 8 9 10 U 12 13 14 15 

.1759 

.8565*** .6806*** 

.1806 .0046 -.6759*** 

.6528*** .4769*** -.2037 .4722** 

-.8009*** -.9769*** -1.6574*** -.9815*** -1.4537*** 

.9491*** .7731*** .0926 .7685*** .2963 1.7500*** 

.5602*** .3843* -.2963 .3796 -.0926 1.3611*** -.3889* 

-,6343*** -.8102*** -1.4907*** -0.8148*** -1.287*** .1667 -1.5833*** -1.1944*** 

.375 .1991 -.4815** .1944 -.2778 1.1759*** -.5741*** -.1852 1.0093*** 

.2963 .1204 -.5602*** .1157 -.3565 1.0972*** -.6528*** -.2639 .9306*** -.0787 

.9954*** .8194*** .1389 .8148*** .3426 1.7963*** .0463 .4352** 1.6296*** .6204*** .6991*** 

.9444*** .7685*** .088 .7639*** .2917 1.7454*** -.0046 .3843* 1.5787*** .5694*** .6481*** -.0509 

1.0741*** .8981*** .2176 .8935*** .4213** 1.875*** .125 .5139*** 1.7083*** .6991*** .7778*** .0787 .1296 

.5417*** .3657 -.3148 .3611 -.1111 1.3426*** -.4074* -.0185 1.1759*** .1667 .2454 -.4537** -.4028 -.5324*** 
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Table 2. Post-hoc comparisons for learning scale by features 


Feature Statistic 

1 Time online Mean 2.9897 


2 Learning partner 


3 Learning 
recommendation for 
course completion 

4 Ratingscales for 
learning material 

5 Timeline for 
achievement of 
objectives 

6 Time for reading and 
task completion 

7 Prompts for self- 
assessment 


8 Further learning 
recommendations 


9 Comparison with 
fellow students 


10 Integration of 
personal schedule 

11 Newsfeed 


12 Repetition of learning 
content 

13 Feedback for 
assignments 

14 Reminder 


15 Term scheduler 


Variance .641 

SO .80049 

Mean 3.1273 

Variance .569 

SO .75453 

Mean 3.6964 

Variance .45 

SO .67053 

Mean 2.7913 

Variance .823 

SO .90699 

Mean 3.6283 

Variance .577 

SO .75945 

Mean 2.4005 

Variance .891 

SO .94404 

Mean 3.7288 

Variance .485 

SO .69613 

Mean 3.4848 

Variance .544 

SO .7374 

Mean 2.6042 

Variance .799 

SO .89386 

Mean 3.2272 

Variance .717 

SO .8465 

Mean 2.7973 

Variance .813 

SO .90176 

Mean 3.5737 

Variance .498 

SD .70591 

Mean 3.3919 

Variance .766 

SD .87541 

Mean 3.0883 

Variance .776 

SD .88097 

Mean 2.788 

Variance .86 

SD .92716 


.13757 


.70668*** .56911*** 


-.19841 -.33598** -.90509*** 


.63856*** .50099*** -.06812 .83697*** 


-.58929*** -.72685*** -1.29597*** -.39087*** -1.22784*** 


.73909*** .60152*** .03241 .93750*** .10053 


.49504*** .35747** -.21164 .69345*** -.14352 


-.38558*** -.52315*** -1.09226*** -.18717 -1.02414*** 


.23743 .09987 -.46925*** .43585*** -.40112*** 


-.19246 -.33003** -.89914*** .00S9S -.83102*** 


.58399*** .44643*** -.12269 .78241*** -.05456 


.40212*** .26455 -.30456* .60053*** -.23644 


.09854 -.03902 -.60813*** .29696* -.54001*** 


-.20172 -.33929** -.90840*** -.00331 -.84028*** 


1.32837*** 


1.08433*** -.24405 


.2037 -1.12467*** -.88062*** 


.82672*** -.50165*** -.25761 .62302*** 


.39683*** -.93155*** -.68750*** .19312 


1.17328*** -.15509 .08896 .96958*** 


.99140*** -.33697** -.09292 .78770*** 


.68783*** -.64054*** -.39649*** .48413*** 


.38757*** -.94081*** -.69676*** .18386 


-.42989*** 


.346S6** .77646*** 


.16468 .59458*** -.18188 


-.13889 .29101* -.48545*** -.30357* 


-.43915*** -.00926 -.78571*** -.60384*** -.30026* 


15 


Note: *** p<.001; ** p<.01; * p<.05 

A hierarchical regression analysis was conducted to determine if students’ evaluation of the features in 
terms of learning, privacy, difficulty, and usefulness are significant predictors if students would like to use a 
certain learning analytics feature. The final regression model (see Table 3) explained a statistically significant 
amount of variance in willingness to use a certain learning analytics feature, A R 2 = .652, F(l, 3235) = 
1518.77, p < 001. Results of the hierarchical regression analysis show that all four variables positively 
predict the willingness to use a certain learning analytics feature. Especially, students’ rating in terms of 
learning and usefulness of a certain learning analytics feature positively predict students’ willingness to use 
it. Accordingly, Hypotheses 3a, 3b, 3c, and 3d are accepted. 

Table 3. Regression analysis predicting willingness to use learning analytics features on learning, privacy, difficulty, and 

usefulness 


Step 1 
learning 
Step 2 
learning 
privacy 
Step 3 
learning 
privacy 
difficulty 
Step 4 
learning 
privacy 
difficulty 
usefulness 
Note. *** p < .001, p < .01, p < .05 


AR 2 

B 

SEB 

P 

.495 

1.011 

.018 

.704*** 

.560 

.942 

.017 

.656*** 


.276 

.013 

.259*** 

.577 

.918 

.017 

.639*** 


.205 

.014 

.193*** 


.170 

.015 

.147*** 

.652 

.676 

.018 

.471*** 


.096 

.013 

.091*** 


.037 

.015 

.032* 


.392 

.015 

.392*** 


R 2 

.496 

.561 

.577 

.653 
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5. DISCUSSION 

As real-time feedback was mentioned from almost all students in the qualitative study, it has presumably 
high relevance for learning, as already postulated by Hattie (2009). Concerning this, learning analytics could 
offer an appropriate approach as the system can provide real-time feedback to each individual learner in 
much more detail than one single teacher could. 

The qualitative results showed that the students prefer learning with printed material and a function to 
document learning activities occurring offline was discussed. This leads to the need to investigate how 
learning offline or conscious informal learning could ideally be entered into a learning analytics system, so 
that invalid analyses due to incomplete data could be decreased and students will not be demotivated only 
because the system did not consider all their learning efforts. As long as most learning takes place outside the 
online learning environment, learning analytics systems can only be considered as an additional service. 
Likewise, the study of Verbert et al. (2014) revealed that learners rated the usefulness of learning analytics 
dashboards low, when many relevant activities happened outside the tracked learning environment. 

Students’ expectations of a learning analytics system, combining several programs and functions would 
allow to tracking their learning behavior in an easier way. Using a PDF annotation program, highlighted 
content and their added thoughts would become more obvious to analyses as well as their paths through 
different programs. By tracking text processing, the emergence of artifacts could be analyzed by the system 
in terms of which further resources might help the student to proceed. 

In the qualitative study students had ambivalent voices in terms of comparisons with fellow students, 
which was also revealed in the quantitative study, as this feature was rated significantly lower in terms of 
willingness to use it than almost all other features (see Table 1 ). 

As the regression analysis showed, students’ evaluation of a learning analytics feature in terms of learning 
positively predicts their willingness to use. Three of the five features, students are the most willing to use, are 
strongly related to support learning and are also evaluated to support learning by the students: repetition of 
learning content, prompts for self-assessment, and further learning recommendations to complete a course. 

The present study shows limitations, as the interviewed students have mainly no experience in using 
learning analytics features, thus it seemed to be difficult to imagine the potential possibilities of big data 
analysis for learning purposes. To control order effects the sequence of the presented learning analytics 
features should be randomized. To consider the dependency of the students’ rating on each feature a bigger 
sample size would be necessary. 

Still there are many open questions in terms of how learning analytics could support learning processes 
and new rose especially from the qualitative study. As some students already mentioned they were 
concerned, if too much support from a learning analytics system might reduce autonomy of learning 
processes, which is related to the components of self-regulated learning. Hence, further research regarding 
the cohesion of learning analytics features and self-regulated learning needs to be initiated to find out if 
learning analytics systems are capable to foster self-regulated learning or if they even hinder it by taking over 
too much of the learners’ responsibility and autonomy (Boekaerts, 1999; Gasevic, Dawson, & Siemens, 
2015). In this respect personalization of learning analytics systems will be of high relevance. 

Within this study the expectations of students were considered, further research needs to take into account 
other stakeholders’ voices and how learning analytics can support self-regulated learning in online learning 
environments by considering learning theoretical assumptions. 


6. CONCLUSION 

From a learning science perspective, the focus of learning analytics should be on understanding and 
supporting learning processes. As learning is a multifaceted and complex process the coherence of learning 
analytics features and learning, especially self-regulated learning needs to be in the focus of educational 
research. The design of valid learning analytics features needs to be based on (self-regulated) learning 
theories and results from qualitative as well as quantitative research. 

However, students expect highly developed learning analytics systems, combining the functions of 
various programs, allowing personalization, showing the results of diverse analyses and giving 
recommendations for further learning. Fortunately, the prerequisite that students are interested and willing to 
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use learning analytics seems to be given as both studies revealed similar results. To meet all stakeholders’ 
expectations and to increase the acceptance and perceived usefulness of learning analytics systems, their 
voices need to be considered beforehand system- wide implementation. 
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